Perception of Statistical Graphics

Susan VanderPlas

05/27/2014

Academic Background

  • Undergrad: Psychology & Applied Math at Texas A&M

  • Bioinformatics at Iowa State (started in 2009)

  • Transferred to Statistics in Fall 2010

  • Finished classes in Spring 2013

Outline

  • Sine Illusion

  • Visual Aptitude and Graphical Inference

  • Hierarchy of Graph Features

Signs of the Sine Illusion

Why We Need to Care

plot of chunk original

Example

8-hour Average Ozone Levels in Houston, TX by temperature at Hobby Airport

plot of chunk example-sineillusion

Example

Residual Ozone Levels in Houston, TX by temperature at Hobby Airport

plot of chunk example-sineillusion2

Explaining the Illusion

The sine illusion results from misapplication of a three-dimensional visual heuristic to ambiguous two-dimensional images Perspective plot of a three-dimensional image similar to the sine illusion

Explaining the Illusion

The sine illusion results from misapplication of a three-dimensional visual heuristic to ambiguous two-dimensional images Perspective plot of the same data, with a vanishing point closer to infinity In this figure, the vanishing point has been moved towards infinity; the lines are straight and closer to the appearance of the sine illusion. The three-dimensional appearance is still intact.

Explaining the Illusion

Context Matters

plot of chunk originalgrid

  • Cleveland and McGill (1984) demonstrated that we can compare the length of two lines accurately
  • Day and Stecher (1991) showed that the illusion persists if there is curvature in the underlying function

Geometry of the Sine Illusion

plot of chunk illusion-geometry

Geometry of the Sine Illusion


  • We perceive the orthogonal width of the implied surface

  • The orthogonal width is a function of the x and y range as well as the aspect ratio of the plot.

  • The perceived orthogonal width is also a function of the slope of the line tangent to the underlying function curve.

Correcting the Illusion

  1. Trend Removal
    Remove the underlying function, plotting the curve and the residuals separately

  1. X Axis Correction
    Reparameterize the x-axis in terms of the slope, so that the absolute slope doesn’t change

  1. Y Axis Correction
    Change the plotted line length (or spread) so that the perceived orthogonal width corresponds to the original (data) line length

Trend Removal

plot of chunk cleveland

Trend Removal

plot of chunk cleveland2

Trend Removal

It is hard to re-create this graphic with separate curves that still provide all of the information

X-axis Transformation

Let \(a\) and \(b\) be the minimum and maximum of the \(x\)-range under consideration.


For any value \(x \in (a,b)\) the following transformation results in a function with constant absolute slope:


\[ (f \circ T)(x) = a + (b-a)\left(\int_{a}^x |f^\prime(z)| dz\right)\left/\left(\int_{a}^{b}|f^\prime(z)| dz\right)\right. \]

X-axis Transformation


plot of chunk xaxisdemo

X-Axis Transformation

Partial Transformations

Shrinkage factor \(w \in (0,1)\): allows a less extreme approach to counteracting the illusion


\[(f \circ T_w)(x) = (1-w) \cdot x + w \cdot (f \circ T)(x)\]


  • \(w=1\) is a full transformation
  • smaller values of \(w\) indicate a less severe correction
  • Under weaker transformations the data more closely reflect the original function \(f(x)\).

X-axis Transformation

plot of chunk xaxisdemoweight

Y-axis Transformation

General Correction

plot of chunk y-generalcorrectioncartoon If we extend the line length so that the extant width matches the original vertical length, our perceptions will match the original data.

Y-axis Transformation

General Correction

The function describing the orthogonal line through \((x_o, f(x_o))\) is given in point-vector form as


\[ {x_o \choose f(x_o)} + \lambda {f^\prime(x_o) \choose 1} \]


for any real-valued \(\lambda\).

Y-axis Transformation

General Correction

Point vector form allows us to solve for \(\lambda\) easily, giving the extant (half) widths as:


\[ |\lambda| \sqrt{1 + f^\prime(x_o)^2} \]


This equation describes the quantity that we perceive rather than the quantity that we want to display (\(\ell/2\))

Y-axis Transformation

General Correction

The general correction factor is thus


\[ \ell/2 \cdot \left(|\lambda| \sqrt{1 + f^\prime(x_o)^2}\right)^{-1} \]


This yields two solutions; one for positive and one for negative values of \(\lambda\) corresponding to upper and lower (half) extant width.

Y-axis Transformation

General Correction

In order to get actual numeric values for \(\lambda\), we need to find end points \(f_1\) and \(f_2\). This system of equations provides solutions for those points:


\[ x - x_o = \lambda f^\prime(x_o) \]


\[ f(x) - f(x_o) = -\lambda \pm \ell/2 \]


Solving these equations requires numerical optimization; we will use linear and quadratic taylor series to simplify the optimization processs.

Y-axis Transformation

Linear and Quadratic Approximations

plot of chunk y-linearcorrectioncartoon

Substituting the endpoints \((x_1, y_1)\) and \((x_2, y_2)\) into the general correction factor produces the linear and quadratic corrections to the sine illusion

Y-axis Transformation

Linear Correction


\[f(x)\approx f(x_0) + (x-x_0) f^\prime(x_0)\]


The correction factor is then


\[\ell_{new}(x_0) = \ell_{old}\sqrt{1+f^\prime(x_0)^2}\]

Y-axis Transformation

Linear Correction

plot of chunk ycorrection1

Y-axis Transformation

Quadratic Correction


\[f(x) \approx f(x_0) + f^\prime(x_0)(x-x_0) + 1/2 f^{\prime\prime}(x_0)(x-x_0)^2\]

The general correction system of equation simplifies to


\[ f^{\prime\prime}(x_0) f^\prime(x_0)^2 \lambda^2 + 2(f^\prime(x_0)^2 + 1) \lambda \pm \ell = 0, \]

Y-axis Transformation

Quadratic Correction

The corrections for the half lengths are then:

\[\begin{eqnarray} \ell_{\text{new}_1}(x_0) &=& 1 /2 \cdot \left(v + \sqrt{ v^2 + f^{\prime\prime}(x_0) f^\prime(x_0)^2\cdot \ell_{\text{old}}}\right) \cdot v^{-1/2} \\ \ell_{\text{new}_2}(x_0) &=& 1 /2 \cdot \left(v + \sqrt{ v^2 - f^{\prime\prime}(x_0) f^\prime(x_0)^2\cdot \ell_{\text{old}}}\right) \cdot v^{-1/2} \end{eqnarray}\]


where \(v = 1 + f^\prime(x_0)^2\)

In the quadratic correction, each half-length is corrected separately, producing a more robust correction

Y-axis Transformation

Quadratic Correction

plot of chunk ycorrection2

Y-axis Transformation

Partial Transformations

The y-axis transformation can be weighted in the same manner as the x-axis transformation.

A Shiny applet was created to explore the x and y corrections.

Testing the Corrections


Goal : Determine the strength of the Sine Illusion by measuring how much correction is required for viewers to say that the lines are of equal length.


A different Shiny applet was created to allow users to manipulate the stimuli using fine-grained adjustments to the weight value.

Participants



  • Using the shiny applet, users could manipulate the weight value presented using -/+ buttons until they were satisfied that the lines were of equal length. The trial was finished when users selected the ‘submit’ button.

Data Collection


  • User identification information: a ‘fingerprint’ consisting of hashed browser and computer characteristics was used to identify unique users

  • IP address localization (34.45.38.XX) provided location information

  • Every user interaction was recorded with a timestamp

  • Trial finished when user clicked either ‘submit’ or ‘skip’ to opt-out of the trial.

Experiment Design

  • 12 (or more) trials, 6 of each correction type

    • Each user completed trials starting at 0 and 1 for both correction types
    • Additional trials were selected using starting weights between 0.25 and 0.75, with point density highest around 0.6
    • After 12 trials, weights were chosen from starting weights between 0 and 1


plot of chunk startingweights

Data Inclusion Criteria

  • Trial recorded at least two user interactions:
    The user must adjust the weight value at least once and then click the submit button.

  • User completed at least 4 trials

  • User selected a weight value that was not severely over-corrected or under corrected (i.e. weight value selected was plausible)

Data Inclusion Criteria

Implausibly Over-corrected and Under-corrected Answers

Answers more extreme than these were excluded from the analysis.

Results


Included Data

Once exclusion criteria were applied, our data consisted of 125 participants who completed 1210 valid trials.

Psychophysics Model

Analysis Idea for Method of Adjustment

Psychophysics Model

Let \(\gamma_X\) represent the optimal weight value for the \(X\)-correction
and \(\gamma_Y\) represent the optimal weight value for the \(Y\) correction.


\(\gamma_\ast = \frac{1}{2}(w_0 + w_1)\)


where \(w_0\) is the preferred weight when starting at 0, and \(w_1\) is the preferred weight when starting at 1.

Psychophysics Model

plot of chunk psychophysics

Random Effects Model

  • \(W_{ij}\) the weight chosen by participant \(i\) on trial \(j\)

  • \[1 \le i \le 125, 1 \le j \le n_i\]
  • \(T(i,j)\) the correction type, where \(T(i,j) \in \{X, Y\}\)

  • Starting weight \(X_{ij}\)

  • \(\alpha_\ast\), the lowest acceptable weight for correction type \(\ast\)
  • \(\beta\), the acceptable weight value interval width
  • Participant-level random intercept \(\gamma_{i, \ast}\)

Random Effects Model


\[ W_{ij} = \alpha_{T(i,j)} + \beta X_{ij} + \gamma_{i, T(i,j)} + \epsilon_{ij}\]


\[\gamma_{iX} \stackrel{\text{ i.i.d.}}{\sim} N(0, \eta_X^2) \ \ \ \ \ \ \ \ \gamma_{iY} \stackrel{\text{ i.i.d.}}{\sim} N(0, \eta_Y^2) \]

\[\epsilon_{ij} \stackrel{\text{ i.i.d.}}{\sim} N(0, \sigma^2) \ \ \ \ \ \ \ \ \text{Cov}(\gamma, \epsilon) = 0\]


The range of acceptable values is

\[(\alpha_\ast, \alpha_\ast + \beta)\]

Random Effects Model


\[ W_{ij} = \alpha_{T(i,j)} + \beta X_{ij} + \gamma_{i, T(i,j)} + \epsilon_{ij}\]


\[\gamma_{iX} \stackrel{\text{ i.i.d.}}{\sim} N(0, \eta_X^2) \ \ \ \ \ \ \ \ \gamma_{iY} \stackrel{\text{ i.i.d.}}{\sim} N(0, \eta_Y^2) \]

\[\epsilon_{ij} \stackrel{\text{ i.i.d.}}{\sim} N(0, \sigma^2) \ \ \ \ \ \ \ \ \text{Cov}(\gamma, \epsilon) = 0\]


We can compare this model to the psychophysics model using the midpoint of this interval, \[\alpha_\ast+\beta/2\]

Random Effects Model

Results
Transformation Threshold Estimate 95% C.I.
X Lower 0.097 (0.048, 0.149)
Upper 0.625 (0.570, 0.684)
Y Lower 0.143 (0.094, 0.187)
Upper 0.671 (0.622, 0.717)

Random Effects Model

plot of chunk ranef

Conclusions

  • Either correction is preferrable to an uncorrected graph

  • Corrections do not have to be fully applied to break the illusion’s power

  • The sine illusion is strong enough to make participants think that lines of unequal length are equal


Importance to Statistical Graphics

We can’t judge variability accurately when there is a nonlinear trend. Knowing is half the battle; having tools to screen for this effect could also be helpful.

Future Work


  • R package with functions to correct data

  • Shiny applet that allows users to upload data and then provides \(x\) and \(y\) corrections

Visual Aptitude and Graphical Inference

Goals

  • Understand what visual skills are related to comprehension of graphics

  • Reduce variability in lineup performance
    • Lineups from Hofmann, Follett, Majumder, & Cook (2012)

  • (Ideally) Link visual skills to specific types of graphics

Visual Search Task

Designed to test participants’ ability to find a target stimulus in a field of distractors

Participants are instructed to find the plot numbered 1-24 which matches the plot labeled “Target”. Participants will complete up to 25 of these tasks in 5 minutes

Paper Folding

Tests participants’ ability to visualize and mentally manipulate figures in three dimensions. Associated with the ability to extrapolate symmetry and reflection over multiple steps.

Participants are instructed to pick the figure matching the sequence of steps shown in the left-hand figure. Participants will complete up to 20 of these tasks in 6 minutes.

Card Rotation

Tests participant’s ability to rotate objects in two dimensions to distinguish between left-hand and right-hand versions of the same figure. Tests spatial reasoning ability and mental rotation skills.

Participants mark each figure on the right hand side as either the same or different than the figure on the left hand side of the dividing line. Participants will complete up to 20 of these tasks (each consisting of 8 figures) in 6 minutes.

Figure Classification

This task is associated with visual reasoning capabilities and we expect that it should correlate with the ability to pick out a signal plot from a lineup.

Participants classify each figure in the second row as belonging to group 1, 2, or 3 (if applicable). Participants will complete up to 14 of these tasks (each consisting of 8 figures to classify) in 8 minutes.

Lineups

Experiment Setup

  • Questionnaire:
    Demographic information, academic background, self-rating of verbal/math/artistic skills, video game usage, etc.
  • Visual Search Task (25 questions)

  • Lineup Task 1:
    boxplots, density plots, dotplots, histograms
  • Card Rotation Task (20 x 8 questions)

  • Lineup Task 2:
    boxplots, violin plots, scatterplot + boxplot overlays
  • Figure Classification Task (14 x 8 questions)

  • Lineup Task 3:
    linear regression and normality assessment, outliers, and residual plots
  • Paper Folding Task (20 questions)

Data Collection


  • Pilot study:
    Working group took the tests (with different lineups) in February
  • Initial Data:
    20 students tested in April
  • More data collection to occur in June/July

Preliminary Results

Preliminary Results

Hierarchy of Graphical Features

Goal

  • Establish a hierarchy of graphical features that are most visually salient

  • Explore which graphs most efficiently present data with regard to salient features

Example Stimuli

Example Stimuli

Challenges

  • Control stimulus strength (i.e. \(r^2\), group size, point density, etc.) to effectively compare color to correlation

  • Appropriate color schemes to maximize sensory distance (while allowing colorblind viewers to perceive the groups)

Timeline

Sine Illusion


  • Paper submitted to JCGS in Oct 2013, revision submitted in March 2014

  • Second paper will likely be submitted to Chance
    • Different user study, same basic conclusion

Timeline

Visual Aptitude


  • Pilot study in Feb 2014
  • Data collection started in April 2014
  • Second round of data collection this summer
  • Paper to be written by September 2014

Timeline

Graph Feature Hierarchy


  • Pilot study in Feb 2014
  • Data collection (Amazon Turk) in September 2014
  • Paper written by November 2014

Timeline




Goal : Defend in January or February 2015